Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(parser) Fix freezing issue with illegal 0 width matches #2524

Merged
merged 4 commits into from
May 3, 2020

Conversation

joshgoebel
Copy link
Member

Closes #2522

@joshgoebel joshgoebel requested review from egor-rogov and allejo May 2, 2020 19:11
@joshgoebel joshgoebel added bug hotfix bad bug, should fix and release ASAP parser labels May 2, 2020
@joshgoebel joshgoebel added this to the 10.0.2 milestone May 2, 2020
@joshgoebel
Copy link
Member Author

joshgoebel commented May 2, 2020

@allejo @egor-rogov Will publish this tomorrow (with 10.0.2) if no one reports an issue until then.

@joshgoebel joshgoebel changed the title fix[parser] add edge case handle for illegal 0 width matches fix(parser) Fix freezing issue with illegal 0 width matches May 3, 2020
@joshgoebel joshgoebel merged commit acbbac8 into highlightjs:master May 3, 2020
@wooorm
Copy link

wooorm commented May 3, 2020

Thanks Josh for the quick fix! ✨

joshgoebel added a commit to joshgoebel/highlight.js that referenced this pull request May 3, 2020
…tjs#2524)

* fix[parser] add edge case handle for illegal 0 width matches
* add last ditch catch all that tries to detect other uncaught freezes
@joshgoebel
Copy link
Member Author

10.0.2 being released now.

Comment on lines +376 to +383
// infinite loops are BAD, this is a last ditch catch all. if we have a
// decent number of iterations yet our index (cursor position in our
// parsing) still 3x behind our index then something is very wrong
// so we bail
if (iterations > 100000 && iterations > match.index * 3) {
const err = new Error('potential infinite loop, way more iterations than matches');
throw err;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I'm late to the party. Quick fix was the right thing to do.
I'd move this check out to the for (;;) loop, in order not to deal with iterations inside processLexeme function.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except then we'd literally have to check it ALL the time... where-as right now it's more of a fallback... but I'll play around with it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But technically your right in that if there were a bug in the above code that this wouldn't be well suited to catch it... :-/

0xflotus added a commit to 0xflotus/highlight.js that referenced this pull request Jun 12, 2020
* (docs) Add Chaos to supported languages (highlightjs#2510)

* fix(parser) fixes sublanguage with no rule matches (highlightjs#2506)

* fix(parser) fixes sublanguage with no rule matches

Resolves highlightjs#2504.

* (chore) Add ESLint config and clean up the major stuff (highlightjs#2503)

* (chore) eslint:recommended
* (chore): eslint_standard
* relax eslint rules for language grammars (to discourage rewriting them in one fell swoop; I'd rather have the blame history intact)
* remove extra escaping
* clean up variables
* more camelcase

* (docs) Add Visual Basic for Applications (VBA) to supported languages (highlightjs#2512)

* (yaml) improve tag support; add verbatim tags (highlightjs#2487)

* YAML parse non-word characters as part of tags
* adds support for verbatim tags

Co-authored-by: Josh Goebel <me@joshgoebel.com>

* fix(javascript/typescript): lambda with parens in parameters fails (highlightjs#2502)

* fix(javascript/typescript): lambda with parens in parameters fails

- Fixes both JavaScript and TypeScript grammars

Fixes samples like:

    const bad = ((a, b) => [...a, b]);
    sides.every((length,width=(3+2+(4/5))) => length > 0 );

This is done by counting parens in the regex that finds arrows
functions. Currently we can only handle 2 levels of nesting as
shown in the second example above.

* allow much richer highlighting inside params
* improve highlighting inside arguments on typescript

* enh(cpp): Improve highlighting of unterminated raw strings

PR highlightjs#1897 switched C++ raw strings to use backreferences, however this
breaks souce files where raw strings are truncated. Like comments, it
would be preferable to highlight them.

- Add `on:begin` and `on:end` to allow more granular matching when
  then end match is dynamic and based on a part of the begin match
- This deprecates the `endSameAsBegin` attribute. That attribute was
  a very specific way to solve this problem, but now we have a much
  more general solution in these added callbacks.

Also related: highlightjs#2259.

Co-authored-by: Josh Goebel <me@joshgoebel.com>

* (chore) C-like uses the new END_SAME_AS_BEGIN mode

* (chore) Ruby uses END_SAME_AS_BEGIN mode/rule

* (parser) make END_SAME_AS_BEGIN a function helper

Adds a mode helper to replace the deprecated `endSameAsBegin` attribute.

The first match group from the begin regex will be compared to the first
match group from the end regex and the end regex will only match if both
strings are identical.

Note this is more advanced functionality than before since now you can
match a larger selection of text yet only use a small portion of it for
the actual "end must match begin" portion.

* (pgsql) add test for $$ quoting existing behavior

- even if that existing behavior is questionable
- the ending span should really close before the $$, not after

Fixing this would involve delving into the sublanguage behavior and I'm
not sure we have time to tackle that right this moment.

* (chore) pgsql uses END_SAME_AS_BEGIN mode/rule now also

* (docs) rename to `mode_reference`; docs for callbacks

- I can never find this file because it's name didn't fully match.
- rename callbacks to `on:begin` and `on:end`

* prevented setter keyword conflicting with setTimeout|setInterval and highlighted them (highlightjs#2514) (highlightjs#2515)

* fix(javascript) prevent setter keyword 'set' conflicting with setTimeout|setInterval (highlightjs#2514)
* enh(javascript) setTimeout|setInterval now highlighted (highlightjs#2514)
* enh (javascript) clearInterval and clearTimeout now highlighted
* add keywords to TypeScript also

* (docs) add TLDR instructions for building and testing

* (dev) improve developer tool UI

* (parser) Build common EMCAscript foundation

Builds a common keyword foundation for any grammar that is
building on top of JavaScript:

- LiveScript
- CoffeeScript
- TypeScript

Also uses this common foundation for JS itself.

* (parser) Adds SHEBANG mode

* (yaml) Add support for inline sequences and mappings (highlightjs#2513)

* Use containers to match inline sequences and mappings
* Add string type for inside inline elements
* Handle nested inline sequences and mappings
* Disallow all braces brackets and commas from strings inside inline mappings or sequences
* clean up implementation
* feed the linter

Co-authored-by: Josh Goebel <me@joshgoebel.com>

* [enh] Add `OPTIMIZE:` and `HACK:` to comment doctags

* (build) browser build is CommonJS and IIFE, no more AMD (highlightjs#2511)

* (build) browser build is CommonJS and IIFE (global) now
* (build) dropping support for AMD, which we never truly supported
  properly in the first place
* (build) add test to make sure browser build works as commonJS module

  Resolves highlightjs#2505

* fix(parser) Fix freezing issue with illegal 0 width matches (highlightjs#2524)

* fix[parser] add edge case handle for illegal 0 width matches
* add last ditch catch all that tries to detect other uncaught freezes

* (docs) added unicorn-rails-log as an 3rd-party language (highlightjs#2528)

- (docs) Add syntax highlighting for Rails Unicorn logging to supported languages.

* (docs) fix supported languages link: it moved again! (highlightjs#2533)

* fix(ts/js) use identifier to match potential keywords (highlightjs#2519)

- (parser) Adds `keywords.$pattern` key to grammar definitions
- `lexemes` is now deprecated in favor of `keywords.$pattern` key
- enh(typescript) use identifier to match potential keywords, preventing false positives 
- enh(javascript) use identifier to match potential keywords, preventing false positives

* fix(javascript) fix regex inside parens after a non-regex (highlightjs#2531)

* make the object attr container smarter
* deal with multi-line comments also
* comments in any order, spanning multiple lines

Essentially makes the object attr container much more sensitive by allowing it to look-ahead thru comments to find object keys - and therefore prevent them from being incorrectly matched by the "value container" rule.

* (parser) Add hljs.registerAlias() public API (highlightjs#2540)

* Add hljs.registerAlias(alias, languageName) public API
* Add .registerAlias() test

* enh(cpp) add `pair`, `make_pair`, `priority_queue` as built-ins (highlightjs#2538)

* (fix) `fixMarkup` would rarely destroy markup when `useBR` was enabled (highlightjs#2532)

* enh(cpp) Recognize `priority_queue`, `pair` as containers (highlightjs#2541)

* chore: rename `registerAlias` to `registerAliases`

Plural form is clearly better as it's not surprising to the reader
to see it being passed an array - where as the singular form might
have been.  Meanwhile it's also easy to assume that it also supports
arrays of any size - including an array with a singular alias.

The fact that it can magically accept a string as the first argument
might not be obvious, but we document it and even if one didn't know
this they could still use the array form of the API without any issue
by passing a one item array.

* (swift) @objcMembers not completely highlighted (highlightjs#2543)

* Fixed @objcMembers in Swift

Would match `@objc` first, and the `Members` part would be unhighlighted

* Update CHANGES.md

* Update swift.js

* (docs) add OCL to list of supported languages (highlightjs#2547)

* (docs) Add Svelte to list of supported languages (highlightjs#2549)

* enh(dart) Add `late` and `required` keywords, and `Never` built-in type (Dart 2.9) (highlightjs#2551)

* Add new Dart 2.9 keywords for Null Safety language feature

* enh(erlang) add support for underscore separators in numeric literals (highlightjs#2554)

* (erlang) add support for underscore separators in numeric literals
* (erlang) add tests

* (docs) add Jolie to Supported Languages (highlightjs#2556)

* (parser/docs) Add jsdoc annotations and TypeScript type file (highlightjs#2517)

Adds JSDoc annotations and a .tsconfig that allows TypeScript to be run in it's "allowJS" mode and apply type and sanity checking to JavaScript code also. See Type Checking JavaScript Files.

I've been using TypeScript a lot lately and finding it very beneficial and wanted to get those same benefits here but without converting the whole project to TypeScript. It was rough at the beginning but now that this is finished I think it's about 80%-90% of the benefits without any of the TS compilation pipeline. The big difference in being JSDoc for adding typing information vs inline types with TypeScript.

Should be super helpful for maintainers using an editor with tight TypeScript integration and the improved docs/comments should help everyone else.

- Adds types/index.d.ts to NPM build (should be useful for TypeScript peeps)
- Improves documentation of many functions
- Adds JSDoc annotations to almost all functions
- Adds JSDoc type annotations to variables that can't be inferred
- Refactors a few smaller things to allow the TypeScript compiler to better infer what 
   is happening (and usually also made the code clearer)

* (parser) highlightBlock result key `re` => `relevance` (highlightjs#2553)

* enh(handlebars) Support for sub-expressions, path-expressions, hashes, block-parameters and literals (highlightjs#2344)

- `htmlbars` grammar is now deprecated. Use `handlebars` instead.

A stub is included so that anyone literally referencing the old `htmlbars` file (say manually requiring it in Node.js, etc) is still covered, but everyone should transition to `handlebars` now.

* fix(typescript) Add missing `readonly` keyword (highlightjs#2562)

* (docs) Mention `c` is a possible class for C (highlightjs#2577)

* fix(groovy) strings are not allowed inside ternary clauses (highlightjs#2565)

* fix(groovy) strings are not allowed inside ternary clauses
* whitespace can also include tabs

* Update @typescript-eslint/parser to the latest version 🚀 (highlightjs#2575)

* chore(package): update @typescript-eslint/parser to version 3.0.0
* chore(package): update lockfile package-lock.json

Co-authored-by: greenkeeper[bot] <23040076+greenkeeper[bot]@users.noreply.github.com>
Co-authored-by: Josh Goebel <me@joshgoebel.com>

* Update @typescript-eslint/eslint-plugin to the latest version 🚀 (highlightjs#2576)

* chore(package): update @typescript-eslint/eslint-plugin to version 3.0.0
* chore(package): update lockfile package-lock.json

Co-authored-by: greenkeeper[bot] <23040076+greenkeeper[bot]@users.noreply.github.com>
Co-authored-by: Josh Goebel <me@joshgoebel.com>

* (parser) properly escape ' and " in HTML output (highlightjs#2564)

* escape quotes also in final HTML output
* [style] update test coding style
* update markup tests with new escaping

This shouldn't be a security issue -- we've always escaped double quotes inside of HTML attribute values (where they could be used to break out of context) - and we've always used double quotes for enclosing attribute values. 

This just goes all the way and now properly escapes quotes everywhere.  Better safe than sorry.

* (docs) add changelog entry for last PR

* add nnfx theme (highlightjs#2571)

* (themes) Add new lioshi theme (highlightjs#2581)

* Added Cisco Command Line to SUPPORTED_LANGUAGES.md (highlightjs#2583)

* (themes) add `nnfx-dark` theme (highlightjs#2584)

* enh(protobuf) Support multiline comments  (highlightjs#2597)

* enh(java) added support for hexadecimal floating point literals (highlightjs#2509)

- Added support for many additional types of floating point literals
- Added related tests

There still may be a few gaps, but this is a pretty large improvement.

Co-authored-by: Josh Goebel <me@joshgoebel.com>

* (chore) Update issue templates (highlightjs#2574)

Co-authored-by: Vladimir Jimenez <allejo@me.com>

* enh(toml)(ini) Improve parsing of complex keys (highlightjs#2595)

Fixes: highlightjs#2594

* (chore) add `.js` extension to import statements (highlightjs#2601)

Adds file extensions to all import specifiers in ./src/ files.  This is useful to run the files straight from source with a web browser , Node.js ESM or Deno.

- Also add eslint rules regarding extensions for imports

* enh(dart) highlight built-in nullable types (highlightjs#2598)

* Dart: allow built-in nullable types with trailing ? to be highlighted

* enh(csharp) highlight generics in more cases (highlightjs#2599)

* (chore) fix tiny style issues, add linting npm task

- fixes tiny style issues
- adds `npm run lint` for linting the main library source
  (not languages which are still much messier)

* (chore) bump dev dependencies

* (chore) upgrade some dev stuff to newer versions

* bump v10.1.0

* (chore) bump copyright

* (chore) more import below metadata comment

Co-authored-by: M. Mert Yıldıran <mehmetmertyildiran@gmail.com>
Co-authored-by: Josh Goebel <me@joshgoebel.com>
Co-authored-by: Hugo Leblanc <dullin@hololink.org>
Co-authored-by: Peter Massey-Plantinga <plantinga.peter@gmail.com>
Co-authored-by: David Benjamin <davidben@google.com>
Co-authored-by: Vania Kucher <dev.kucher@gmail.com>
Co-authored-by: SweetPPro <sweetppro@users.noreply.github.com>
Co-authored-by: Alexandre ZANNI <16578570+noraj@users.noreply.github.com>
Co-authored-by: Taufik Nurrohman <t.nurrohman77@gmail.com>
Co-authored-by: Lin <50829219+Linhk1606@users.noreply.github.com>
Co-authored-by: nicked <nicked@gmail.com>
Co-authored-by: Nicolas Homble <nhomble@terpmail.umd.edu>
Co-authored-by: Ryandi Tjia <ryandi.tjia@me.com>
Co-authored-by: Sam Rawlins <srawlins@google.com>
Co-authored-by: Sergey Prokhorov <seriy.pr@gmail.com>
Co-authored-by: Brian Alberg <brian@alberg.org>
Co-authored-by: Nils Knappmeier <github@knappi.org>
Co-authored-by: Martin <7252614+Lhoerion@users.noreply.github.com>
Co-authored-by: Derek Lewis <DerekNonGeneric@inf.is>
Co-authored-by: greenkeeper[bot] <23040076+greenkeeper[bot]@users.noreply.github.com>
Co-authored-by: Jim Mason <jmason@ibinx.com>
Co-authored-by: lioshi <lionel.fenneteau@gmail.com>
Co-authored-by: BMatheas <65114274+BMatheas@users.noreply.github.com>
Co-authored-by: Pavel Evstigneev <pavel.evst@gmail.com>
Co-authored-by: Vladimir Jimenez <allejo@me.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: TupikovVladimir <vladimir.tupikov@devexpress.com>
joshgoebel added a commit that referenced this pull request Jul 29, 2020
This was referenced Mar 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug hotfix bad bug, should fix and release ASAP parser
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Infinite loop for illegal tokens when ignore_illegals or highlightAuto
3 participants